Three-dimensional surface profile imaging method and apparatus using single spectral light condition

ABSTRACT

A three-dimensional surface profile imaging method and system uses a single spectral light illumination constraint to guarantee consistent RGB values corresponding to a given light spectrum, regardless of the surface reflectance characteristics of the object being imaged. In one embodiment, each light sheet projected onto the object contains only a single wavelength. As a result, the spectral composition of the projected light at any surface point of the object will be independent of the light intensity and the reflectance characteristics of the surface.

REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Appln. No.60/178,695, filed Jan. 28, 2000, the disclosure of which is incorporatedherein by reference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention is directed to three-dimensional surface profileimaging, and more particularly to a method and apparatus forthree-dimensional imaging that uses color ranging to conduct surfaceprofile measurement.

2. Description of the Related Art

A three dimensional surface profile imaging method and apparatusdescribed in U.S. Pat. No. 5,675,407 (“the '407 patent”), the disclosureof which is incorporated herein by reference in its entirety, conductsimaging by projecting light through a linear variable wavelength filter(LVWF), thereby projecting light having a known, spatially distributedwavelength spectrum on the objects being imaged. The LVWF is arectangular optical glass plate coated with a color-filtering film thatgradually varies in color, (i.e., wavelength). If the color spectrum ofa LVWF is within the visible light region, one edge of the filterrectangle may correspond to the shortest visible wavelength (i.e. blueor violet) while the opposite edge may correspond to the longest visiblewavelength, (i.e. red). The wavelength of light passing through thecoated color-filtering layer is linearly proportional to the distancebetween the position on the filter glass where the light passes and theblue or red edge. Consequently, the color of the light is directlyrelated to the angle θ, shown in FIG. 1, at which the light leaves therainbow projector and LVWF.

Referring to FIGS. 1 and 2 in more detail, the imaging method andapparatus is based on the triangulation principle and the relationshipbetween a light projector 100 that projects through the LVWF 101, acamera 102, and the object or scene being imaged 104. As shown in FIG.1, a triangle is uniquely defined by the angles theta (θ) and alpha (α),and the length of the baseline (B). With known values for θ, α, and B,the distance (i.e., the range R) between the camera 102 and a point Q onthe object's surface can be easily calculated. Because the baseline B ispredetermined by the relative positions of the light projector 100 andthe camera 102, and the value of α can be calculated from the camera'sgeometry, the key to the triangulation method is to determine theprojection angle, θ, from an image captured by the camera 102 and moreparticularly to determine all θ angles corresponding to all the visiblepoints on an object's surface in order to obtain a full-frame 3D imagein one snapshot.

FIG. 2 is a more detailed version of FIG. 1 and illustrates the mannerin which all visible points on the object's surface 104 is obtained viathe triangulation method. As can be seen in the Figure, the lightprojector 100 generates a fan beam of light 200. The fan beam 200 isbroad spectrum light (i.e., white light) which passes through the LVWF101 to illuminate one or more three-dimensional objects 104 in the scenewith a pattern of light rays possessing a rainbow-like spectrumdistribution. The fan beam of light 200 is composed of multiple verticalplanes of light 202, or “light sheets”, each plane having a givenprojection angle and wavelength. Because of the fixed geometricrelationship among the light source 100, the lens of the camera 102, andthe LVWF 101, there exists a one-to-one correspondence between theprojection angle (θ) of the vertical plane of light and the wavelength(λ) of the light ray. Note that although the wavelength variations areshown in FIG. 2 to occur from side to side across the object 104 beingimaged, it will be understood by those skilled in the art that thevariations in wavelength could also be made from top to bottom acrossthe object 104 or scene being imaged.

The light reflected from the object 104 surface is then detected by thecamera 102. If a visible spectrum range LVWF (400-700 nm) is used, thecolor detected by the camera pixels is determined by the proportion ofits primary color Red, Green, and Blue components (RGB). The colorspectrum of each pixel has a one-to-one correspondence with theprojection angle (θ) of the plane of light due to the fixed geometry ofthe camera 102 lens and the LVWF 101 characteristics. Therefore, thecolor of light received by the camera 102 can be used to determine theangle θ at which that light left the light projector 100 through theLVWF 101.

As described above, the angle α is determined by the physicalrelationship between the camera 102 and the coordinates of each pixel onthe camera's imaging plane. The baseline B between the camera's 102focal point and the center of the cylindrical lens of the lightprojector 100 is fixed and known. Given the value for angles α and θ,together with the known baseline length B, all necessary information isprovided to easily determine the full frame of three-dimensional rangevalues (x,y,z) for any and every visible spot on the surface of theobjects 104 seen by the camera 102.

As shown in FIG. 3, given the projection angle θ, the three-dimensionalalgorithm for determining the (x,y,z) coordinates of any surface spotQ(x,y,z) on a three-dimensional object is given below based on thefollowing triangulation principle: $\begin{matrix}{{x = {\frac{B}{{f*{ctg}\quad \theta} - u}*u}},{y = {\frac{B}{{f*{ctg}\quad \theta} - u}*v}},{z = {\frac{B}{{f*{ctg}\quad \theta} - u}*f}}} & (1)\end{matrix}$

As a result, the three-dimensional imaging system described above cancapture full-frame, high spatial resolution three-dimensional imagesusing a standard camera, such as a charge coupled device camera, in realtime without relying on any moving parts. Further, because the imagingsystem does not rely on a laser, it does not pose any hazard to the eyeswhen used in clinical applications. Also, because the wavelength of thelight projected onto the object surface continuously varies, there is notheoretical limitation on the measurement accuracy that can be achievedby the system. The actual accuracy of a specific system will depend onsystem implementation and will be affected primarily by limiting factorssuch as the optical system design, the quality and resolution of thecamera, light spectral emission of the light source projector; noiselevel and resolution of the frame grabber, calibration algorithms, andthe three-dimensional imaging processing algorithms.

To avoid allowing the surface color of the object being imaged fromaffecting the imaging results, the system may obtain an image of theobject under normal light conditions before projecting the filteredlight onto the object. The image obtained under normal light conditionsis then subtracted from the image obtained under LVWF light conditionsto eliminate the effects of the object color on the image.

Even when the system compensates for the color of the object, however,the consistency of the spectral power distribution and the RGB value ofeach pixel may vary when light is projected onto the object through theLVWF based on the reflection characteristics of the object's surface,particularly if the object is not white and/or not uniformly colored.

There is a need for a surface profile imaging method and apparatus thatis able to generate consistent RGB values regardless of the reflectioncharacteristics of the surface being imaged.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a method and apparatusfor three-dimensional surface imaging that avoids variations in the RGBvalue of each pixel due to the reflection characteristics of theobject's surface. More particularly, a light source in the systemilluminates an object or scene with a light pattern having a spatiallyvarying wavelength and composed of at least one light plane. The lightplane corresponds to at least one angle at which the light of thatwavelength is emitted and contains only a single spectral component.

By imposing a single spectral light condition on the light source, theRGB values of each pixel will be independent of the light intensity ofthe light source and the reflectance characteristics of the object orscene being imaged. As a result, any color matching function that isconducted to link the wavelength of the light projected on the object orscene at a given point and that point's position will be consistent,regardless of the color and/or reflectance characteristics of theobject's or scene's surface.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified diagram illustrating a triangulation principleused in the present invention;

FIG. 2 is a representative diagram of the components used by theinventive system; and

FIG. 3 is a plan view of the system shown in FIG. 2.

DESCRIPTION OF THE PREFERRED EMBODIMENT

Generally, the inventive system and method system ensures that the RGBvalues corresponding to a given spectral composition f(λ) areconsistent, regardless of the reflectance characteristics of the surfacebeing imaged. By way of background, the invention recognizes that thespectral composition f(λ) of the light source 100 is a parameter that iscontrollable in the system shown in FIG. 2. Because this parameter iscontrollable, imposing a restrictive condition on f(λ) can make the RGBvalues consistent for f(λ), regardless of r(λ) and s(λ). This will beexplained in greater detail below.

In a monochrome camera, the pixel values are computed from the integralof the spectral power distribution (SPD) of the light measured by acharge coupled device (CCD) element in the camera 102. If the camera 102receives light directly from a light source, a given pixel in the CCDelement of the camera 102 generates output value p by summing the amountof light f(λ) received by the pixel at each wavelength, weighted by theresponsiveness of the camera pixel to a given wavelength, s(λ). That is:$\begin{matrix}{p = {\int_{\lambda_{\min}}^{\lambda_{\max}}{{f(\lambda)}{s(\lambda)}\quad {\lambda}}}} & (2)\end{matrix}$

where f(λ) is the spectral composition (spectral power distribution) ofillumination source; s(λ) is the spectral sensitivity function of thesensor; and λ max and λ min are the upper and lower bounds of thewavelengths visible to CCD pixels in the CCD elements.

When the camera 102 is used to obtain images of real world objects,where the light source position may not be controllable, the reflectancecharacteristics of the surface of the objects will affect the pixelvalues because the camera 102 will not receive light directly from thelight source in most real-world cases. The pixel value p in this case isa integral of the amount of light f(λ) received by the pixel of thecamera at each wavelength, weighted by the surface reflectance functionof the object, r(λ), weighted further by the responsiveness of thecamera pixel to various wavelength, s(λ). That is: $\begin{matrix}{p = {\int_{\lambda_{\min}}^{\lambda_{\max}}{{f(\lambda)}{r(\lambda)}{s(\lambda)}\quad {\lambda}}}} & (3)\end{matrix}$

where r(λ) is the surface spectral reflectance function of the object inthe scene 104.

If the camera 102 is a CCD camera rather than a monochrome camera, colorfilters are interposed between the incoming illumination and the pixelelement of the camera 102. Each filter has a transmittance functionτ(λ), specifying the fraction of light transmitted at each wavelength.Thus, the pixel value is specified by the above integral with s(λ)replaced by τ(λ) s(λ). Typically, three color filters (Red, Green, andBlue) are used in a color CCD camera with transmittance τ_(r)(λ),τ_(g)(λ), and τ_(b)(λ), respectively. More particularly, the respectivespectral sensitivity functions of the sensor for each primary color isas follows:

s _(r)(λ)=τ_(r)(λ)s(λ), s _(g)(λ)=τ_(g)(λ)s(λ), s_(b)(λ)=τ_(b)(λ)s(λ)  (4)

As a result, the RGB value of each pixel of a color camera can becomputed by the expression: $\begin{matrix}{\begin{bmatrix}R \\G \\B\end{bmatrix} = \begin{bmatrix}{\int_{\lambda_{\min}}^{\lambda_{\max}}{{f(\lambda)}{r(\lambda)}{s_{r}(\lambda)}\quad {\lambda}}} \\{\int_{\lambda_{\min}}^{\lambda_{\max}}{{f(\lambda)}{r(\lambda)}{s_{g}(\lambda)}\quad {\lambda}}} \\{\int_{\lambda_{\min}}^{\lambda_{\max}}{{f(\lambda)}{r(\lambda)}{s_{b}(\lambda)}\quad {\lambda}}}\end{bmatrix}} & (5)\end{matrix}$

Equation (5) is a general formula of image formation and pixel valuecomputation of color CCD cameras. More particularly, equation (5)illustrates the relationship among the reflection characteristics ofobject surface, sensitivity function of CCD pixels, light spectrum ofactive illumination source, and the signal output of the camera as wellas shows the effect of the surface reflectance function r(λ) plays onthe CCD image formation.

As can be seen in equation (5), the RGB value of a CCD pixel is anintegral function of three spectral power distribution functions, i.e.,spectral composition of active illumination source f(λ), surfacereflectance function r(λ), and sensitivity function of the CCD camerapixel s(λ). Even if r(λ) and s(λ) are known, it is generally notpossible to uniquely determine f(λ) based on a single set of RGB valuesto determine the projection angle corresponding to each pixel. Further,in many applications, information about the surface reflectance r(λ) isnot available, making determination of f(λ) even more unlikely.

Note, however, that it is not necessary when using a LVWF 101 in theimaging system to recover explicitly the function form of f(λ) based onRGB values, due to the color matching scheme used to determine theprojection angle. Instead, the detected light spectrum is compared withvalues pre-stored in a look-up table in the camera's processor or in aseparate processor coupled to the camera to get a correspondingprojection angle. In this case, the spectrum of projected light is onlyused as an information carrier for the projection angle of the lightsheet. As long as the CCD camera can provide the same RGB values for thesame projected light illumination (with various reflectance functionsr(λ)), the specific form of the function f(λ) is not important. Instead,consistency of RGB values corresponding to light spectrum of f(λ),regardless of the surface reflectance function r(λ), has greaterimportance because the consistency facilitates color matching.

With both f(λ) and r(λ) being freely variable, however, the consistencyof RGB values corresponding to f(λ) is difficult to achieve, therebymaking color matching difficult, particularly because differences in thesurface characteristics (e.g., surface color, reflectivity of thesurface material, non-uniform surface color, etc.) will change the RGBvalues.

To restrict the spectral composition f(λ), the inventive system imposesa “single spectral light condition” to the light projector 100 toguarantee the consistency of RGB values corresponding to light spectrumof f(λ), regardless of the surface reflectance function r(λ). Under thiscondition, each individual light sheet projected by the light projector100 contains only a single wavelength spectral component. Moreparticularly, the spectral composition of the projected light at anysingle surface point on the object 104 being imaged can be expressed bythe following function:

f(λ)=m ₀δ(λ−λ₀)  (6)

where m₀ is a variable indicating the magnitude of the light intensityand λ₀ is the wavelength of the single wavelength light impinged on agiven surface spot.

Under this single spectral light illumination condition, the normalizedRGB values (r,g,b) of a color camera are independent of the intensity ofthe spectral light m₀ as well as the surface reflectance function r(λ)as long as they are both non-zero. More particularly, under the singlespectral light illumination condition, the spatially varying wavelengthlight source from the light projector 100 shown in FIG. 2 produces onlysingle spectral light at each surface point on the object being imaged.Due to the property of the delta function of a single wavelengthspectral light shown in equation (6), the pixel value p for each pixelin the generated image is:

p=m ₀ r(λ₀)s(λ₀)  (7)

where λ₀ is the wavelength of the single wavelength light impinged onthe surface spot observed by the CCD pixel. Based on equation (7), theR, G, and B values of each pixel therefore can be expressed as:$\begin{matrix}{\begin{bmatrix}R \\G \\B\end{bmatrix} = {m_{0}{{r\left( \lambda_{0} \right)}\quad\begin{bmatrix}{s_{r}\left( \lambda_{0} \right)} \\{s_{g}\left( \lambda_{0} \right)} \\{s_{b}\left( \lambda_{0} \right)}\end{bmatrix}}}} & (8)\end{matrix}$

The wavelength information of projected light is embedded in thenormalized RGB values of each pixel. Assuming that both m₀ and r(λ₀) arenon-zero, the normalized RGB values are expressed as follows:$\begin{matrix}{\begin{bmatrix}r \\g \\b\end{bmatrix} = {{\frac{1}{\sqrt{R^{2} + G^{2} + B^{2}}}\begin{bmatrix}R \\G \\B\end{bmatrix}} = {\frac{1}{\sqrt{{s_{r}\left( \lambda_{0} \right)}^{2} + {s_{g}\left( \lambda_{0} \right)}^{2} + {s_{b}\left( \lambda_{0} \right)}^{2}}}\begin{bmatrix}{s_{r}\left( \lambda_{0} \right)} \\{s_{g}\left( \lambda_{0} \right)} \\{s_{b}\left( \lambda_{0} \right)}\end{bmatrix}}}} & (9)\end{matrix}$

As can be seen in equation (9), the normalized RGB values, r, g, and bare independent of m₀ and r(λ₀).

Under the single spectrum light condition, then, the wavelengthinformation of reflected light captured by a camera 102 in the form ofnormalized RGB values depends solely on the sensitivity function of thecamera 102, regardless of the light intensity and surface reflectancecharacteristics of the object 104 being imaged. The sensitivity functionof the camera 102, which indicates the responsiveness of a given camerapixel to various wavelengths, can be obtained from the cameramanufacturer or through calibration procedures. Based on the normalizedRGB data and the known sensitivity function of a camera 102, one canuniquely determine the corresponding wavelength of the light projectedonto the object 104. The three-dimensional characteristics of the objectare then obtained as explained in, for example, U.S. Pat. No. 5,675,407to obtain the three-dimensional range information of the object 104.Note that because of the single spectral light condition restriction,the color matching process used to obtain the surface information willwork, regardless of whether the object surfaces are pure white,non-white, uniformly colored, mixed colored, or even speckled.

Note that although the above description focuses on using visible lightfor conducting the three-dimensional surface imaging, infrared (IR) orultra-violet (UV) light sources with suitable wavelength sensitivedetectors can also be used for special applications with minimalmodification of the imaging system. These applications will not bespecifically addressed herein, but will be readily practicable by thoseskilled in the art based on the principles disclosed herein.

While the invention has been specifically described in connection withcertain specific embodiments thereof, it is to be understood that thisis by way of illustration and not of limitation, and the scope of theappended claims should be construed as broadly as the prior art willpermit.

What is claimed is:
 1. A device for creating a three-dimensional profileof an object or scene being imaged, the device comprising: a lightsource for illuminating said object or scene with a light pattern havingat least one light plane, wherein said light pattern varies inwavelength spatially across said object or scene, and wherein said atleast one light plane corresponds to at least one angle at which lightof that wavelength is emitted by said light source and contains only asingle spectral component; a camera for imaging said object or scene asilluminated with said light pattern; and a processor for calculating adistance to a point on said object or in said scene based on a baselinedistance between said light source and said camera, an angle betweensaid camera and said baseline, and an angle at which light striking thepoint is emitted by said light source as determined from red/green/blue(RGB) values corresponding to the wavelength of the light striking thepoint.
 2. The device of claim 1, wherein the single spectral componentis based on a light intensity value of the light source and thewavelength emitted by said light source.
 3. The device of claim 2,wherein the RGB values depend on a sensitivity function of the cameraand are independent of the light intensity of the light source and asurface reflectance of the object or scene.
 4. The device of claim 1,wherein said light pattern varies over a visible light spectrum.
 5. Thedevice of claim 1, wherein said light pattern varies over an infraredlight spectrum.
 6. The device of claim 1, wherein said light patternvaries over an ultraviolet light spectrum.
 7. The device of claim 1,wherein the processor calculates the distance to the point bytriangulation of the baseline distance between said light source andsaid camera, the angle between said camera and said baseline, and theangle at which light striking the point is emitted by said light source.8. A device for creating a three-dimensional profile of an object orscene being imaged, the device comprising: a light source forilluminating said object or scene with a light pattern having aplurality of light planes, wherein said light pattern varies inwavelength spatially across said object or scene, and wherein each ofsaid plurality of light planes corresponds to at least one angle atwhich light of that wavelength is emitted by said light source andcontains only a single spectral component, the single spectral componentbeing based on a light intensity value of the light source and thewavelength emitted by said light source for one of said light planes; acamera for imaging said object or scene as illuminated with said lightpattern; and a processor for calculating a distance to a point on saidobject or in said scene using triangulation based on a baseline distancebetween said light source and said camera, an angle between said cameraand said baseline, and an angle at which light striking the point isemitted by said light source as determined from red/green/blue (RGB)values corresponding to the wavelength of the light striking the point,wherein the RGB values depend on a sensitivity function of the cameraand are independent of the light intensity of the light source and asurface reflectance of the object or scene.
 9. The device of claim 8,wherein said light pattern varies over a visible light spectrum.
 10. Thedevice of claim 8, wherein said light pattern varies over an infraredlight spectrum.
 11. The device of claim 8, wherein said light patternvaries over an ultraviolet light spectrum.
 12. A method for creating athree-dimensional profile of an object or scene being imaged, the methodcomprising the steps of: illuminating the object or scene with a lightpattern having a plurality of light planes, wherein said light patternvaries in wavelength spatially across said object or scene, and whereinsaid at least one light plane corresponds to at least one angle at whichlight of that wavelength is emitted and contains only a single spectralcomponent; imaging said object or scene as illuminated in saidilluminating step; calculating a distance to a point on said object orin said scene using triangulation based on a baseline distance betweensaid light source and said camera, an angle between said camera and saidbaseline, and an angle at which light striking the point is emitted bysaid light source as determined from red/green/blue (RGB) valuescorresponding to the wavelength of the light striking the point.
 13. Themethod of claim 12, wherein the RGB values depend on a sensitivityfunction of the camera and are independent of the light intensity of thelight source and a surface reflectance of the object or scene.
 14. Themethod of claim 12, wherein the illuminating step varies the lightpattern over a visible light spectrum.
 15. The method of claim 12,wherein the illuminating step varies the light pattern over an infraredlight spectrum.
 16. The method of claim 12, wherein the illuminatingstep varies the light pattern over an ultraviolet light spectrum.